In recent years, a number of studies regarding a mixed reality (“MR”) have been vigorously made.
For the MR, there are provided a video see-through method in which an image in a real space photographed by a photographing apparatus such as a video camera is displayed with an image in a virtual space (e.g., a virtual object or the character information drawn by the computer graphics (hereinafter referred to as CG)) superposed thereon, and an optical see-through method in which an image in the real space is optically transmitted to an HMD (Head-Mounted Display) mounted on the user's head and displayed with an image in the virtual space superposed on a display screen of the HMD.
The applications of MR include the uses of medical assistance for presenting the interior of the patient's body to the physician as seeing through and the uses of work assistance by displaying the real things with an assembling procedure of products in the factory superposed thereon. Therefore, some new fields quite different in quality from the prior VR are expected.
A common requirement for these applications is a technique for making alignment between the real space and the virtual space, for which various attempts have been made up to now.
An alignment problem with the MR of video see-through method results in the problem of correctly attaining the positional attitude at the visual point of the photographing apparatus. Also, an alignment problem with the MR of optical see-through method similarly results in the problem of obtaining the positional attitude at the user's visual point.
In the conventional MR system (particularly an indoor MR system), a method of solving these problems typically involves deriving the positional attitude at the visual point, using a positional attitude sensor such as a magnetic sensor or a ultrasonic sensor.
On one hand, the conventional MR system for outdoors makes use of a gyro sensor to derive the attitude at the visual point (more strictly, a three-axis attitude sensor consisting of a combination of a plurality of gyro sensors for measuring the angular velocities in the three axis directions and a plurality of acceleration sensors for measuring the accelerations in the three axis directions, as a matter of convenience, called a gyro sensor in this specification).
However, when a gyro sensor is used to obtain the attitude at the visual point, the gyro sensor of high precision has a drift error, so that a measurement error will occur in the azimuth direction along with the elapse of time. Also, the gyro sensor is only able to make the attitude measurements, and can not follow the changes in the visual point position. In other words, there may occur some dislocation between the real space and the virtual space along with the elapse of time or the changes in the visual point position.